{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tJTHvE8Qe5nM"
   },
   "source": [
    "# Ungraded Lab: Exploring Convolutions\n",
    "\n",
    "In this lab, you will explore how convolutions work by creating a basic convolution on a 2D grayscale image. Start by importing everything you will need in this lab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# Ascent image that you will work with from the SciPy library\n",
    "from scipy.datasets import ascent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, you will load the [ascent](https://docs.scipy.org/doc/scipy/reference/generated/scipy.datasets.ascent.html) image from [SciPy](https://scipy.org/). It's a nice, built-in picture with lots of angles and lines. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 784,
     "status": "ok",
     "timestamp": 1639058947063,
     "user": {
      "displayName": "Chris Favila",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "17311369472417335306"
     },
     "user_tz": -480
    },
    "id": "DZ5OXYiolCUi"
   },
   "outputs": [],
   "source": [
    "# load the ascent image\n",
    "ascent_image = ascent()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SRIzxjWWfJjk"
   },
   "source": [
    "You can use the pyplot library to show the image so you'll know what it looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 248
    },
    "executionInfo": {
     "elapsed": 976,
     "status": "ok",
     "timestamp": 1639059000048,
     "user": {
      "displayName": "Chris Favila",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "17311369472417335306"
     },
     "user_tz": -480
    },
    "id": "R4p0cfWcfIvi",
    "outputId": "4565e085-4fb0-4129-8e83-ee4dc6646250"
   },
   "outputs": [],
   "source": [
    "# Visualize the image\n",
    "plt.grid(False)\n",
    "plt.gray()\n",
    "plt.axis('off')\n",
    "plt.imshow(ascent_image)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "C1mhZ_ZTfPWH"
   },
   "source": [
    "The image is stored as a numpy array so you can create the transformed image by first copying that array. You can also get the dimensions of the image so you can loop over it later. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 353,
     "status": "ok",
     "timestamp": 1639059122348,
     "user": {
      "displayName": "Chris Favila",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "17311369472417335306"
     },
     "user_tz": -480
    },
    "id": "o5pxGq1SmJMD"
   },
   "outputs": [],
   "source": [
    "# Copy image to a numpy array\n",
    "image_transformed = np.copy(ascent_image)\n",
    "\n",
    "# Get the dimensions of the image\n",
    "size_x = image_transformed.shape[0]\n",
    "size_y = image_transformed.shape[1]\n",
    "\n",
    "print(f\"image_transformed has shape: {size_x, size_y}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y7PwNkiXfddd"
   },
   "source": [
    "Now you can create a filter as a 3x3 array. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 544,
     "status": "ok",
     "timestamp": 1639059236890,
     "user": {
      "displayName": "Chris Favila",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "17311369472417335306"
     },
     "user_tz": -480
    },
    "id": "sN3imZannN5J"
   },
   "outputs": [],
   "source": [
    "# Experiment with different values and see the effect\n",
    "filter = [[0, 1, 0], [1, -4, 1], [0, 1, 0]]\n",
    "\n",
    "# A couple more filters to try for fun!\n",
    "# filter = [ [-1, -2, -1], [0, 0, 0], [1, 2, 1]]\n",
    "# filter = [ [-1, 0, 1], [-2, 0, 2], [-1, 0, 1]]\n",
    "\n",
    "# If all the digits in the filter don't add up to 0 or 1, you \n",
    "# should probably do a weight to get it to do so\n",
    "# so, for example, if your weights are 1,1,1 1,2,1 1,1,1\n",
    "# They add up to 10, so you would set a weight of .1 if you want to normalize them\n",
    "weight  = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JQmm_iBufmCz"
   },
   "source": [
    "Now you can create a convolution. You will iterate over the image, leaving a 1 pixel margin, and multiplying each of the neighbors of the current pixel by the value defined in the filter (i.e. the current pixel's neighbor above it and to the left will be multiplied by the top left item in the filter, etc.) \n",
    "\n",
    "You'll then multiply the result by the weight, and then ensure the result is in the range 0-255.\n",
    "\n",
    "Finally you'll load the new value into the transformed image.\n",
    "\n",
    "*Note: The main objective of this notebook is to simply demonstrate the effect of convolutions and pooling visually. Thus, you will skip some optimizations like dropping border pixels after convolutions, handling odd image dimensions in max pooling, etc.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 3511,
     "status": "ok",
     "timestamp": 1639059241813,
     "user": {
      "displayName": "Chris Favila",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "17311369472417335306"
     },
     "user_tz": -480
    },
    "id": "299uU2jAr90h"
   },
   "outputs": [],
   "source": [
    "# Iterate over the image\n",
    "for x in range(1, size_x-1):\n",
    "    for y in range(1, size_y-1):\n",
    "        convolution = 0.0\n",
    "        convolution = convolution + (ascent_image[x-1, y-1] * filter[0][0])\n",
    "        convolution = convolution + (ascent_image[x-1, y] * filter[0][1])  \n",
    "        convolution = convolution + (ascent_image[x-1, y+1] * filter[0][2])     \n",
    "        convolution = convolution + (ascent_image[x, y-1] * filter[1][0])    \n",
    "        convolution = convolution + (ascent_image[x, y] * filter[1][1])    \n",
    "        convolution = convolution + (ascent_image[x, y+1] * filter[1][2])    \n",
    "        convolution = convolution + (ascent_image[x+1, y-1] * filter[2][0])    \n",
    "        convolution = convolution + (ascent_image[x+1, y] * filter[2][1])    \n",
    "        convolution = convolution + (ascent_image[x+1, y+1] * filter[2][2])    \n",
    "      \n",
    "        # Multiply by weight\n",
    "        convolution = convolution * weight   \n",
    "      \n",
    "        # Check the boundaries of the pixel values\n",
    "        if(convolution < 0):\n",
    "            convolution = 0\n",
    "        if(convolution > 255):\n",
    "            convolution = 255\n",
    "\n",
    "        # Load into the transformed image\n",
    "        image_transformed[x, y] = convolution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6XA--vgvgDEQ"
   },
   "source": [
    "After the loop, you can now plot the image to see the effect of the convolution!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 269
    },
    "executionInfo": {
     "elapsed": 899,
     "status": "ok",
     "timestamp": 1639059523867,
     "user": {
      "displayName": "Chris Favila",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "17311369472417335306"
     },
     "user_tz": -480
    },
    "id": "7oPhUPNhuGWC",
    "outputId": "2aee35d3-e378-441c-e497-1c215722c34c"
   },
   "outputs": [],
   "source": [
    "# Plot the image. Note the size of the axes -- you will compare this later after max pooling\n",
    "plt.gray()\n",
    "plt.grid(False)\n",
    "plt.imshow(image_transformed)\n",
    "plt.show()   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xF0FPplsgHNh"
   },
   "source": [
    "## Effect of Max Pooling\n",
    "\n",
    "The next cell will show a (2, 2) pooling. The idea here is to iterate over the image, and look at the pixel and it's immediate neighbors to the right, beneath, and right-beneath. It will take the largest of them and load it into the new image. Thus, the new image will be 1/4 the size of the old -- with the dimensions on X and Y being halved by this process. You'll see that the features get maintained despite this compression!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Assign dimensions half the size of the original image\n",
    "new_x = int(size_x / 2)\n",
    "new_y = int(size_y / 2)\n",
    "\n",
    "# Create blank image with reduced dimensions\n",
    "newImage = np.zeros((new_x, new_y))\n",
    "\n",
    "# Iterate over the image\n",
    "for x in range(0, size_x, 2):\n",
    "    for y in range(0, size_y, 2):\n",
    "    \n",
    "        # Store all the pixel values in the (2,2) pool\n",
    "        pixels = []\n",
    "        pixels.append(image_transformed[x, y])\n",
    "        pixels.append(image_transformed[x+1, y])\n",
    "        pixels.append(image_transformed[x, y+1])\n",
    "        pixels.append(image_transformed[x+1, y+1])\n",
    "\n",
    "        # Get only the largest value and assign to the reduced image\n",
    "        newImage[int(x / 2), int(y / 2)] = max(pixels)\n",
    "\n",
    "# Plot the image. Note the size of the axes -- it is now half of the image loaded earlier\n",
    "plt.gray()\n",
    "plt.grid(False)\n",
    "plt.imshow(newImage)\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "C1_W3_Lab_2_exploring_convolutions.ipynb",
   "provenance": [
    {
     "file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/12_sep_2021_fixes/C1/W3/ungraded_labs/C1_W3_Lab_2_exploring_convolutions.ipynb",
     "timestamp": 1639058610295
    }
   ]
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
